Using Facial Expressions for Personalised Gaming

Artificial intelligence (AI) in games ideally provides satisfactory game experiences for all players, regardless of gender, age, capabilities, or experience [4]: it personalises games, i.e., the game experience is continuously tailored to fit the individual player. A major challenge in game personalisation is how to build and estimate a model of how game elements affect the player. In particular, we are interested in which elements have a positive effect on the affective state of the player. However, effectively estimating information about this affective state from the information that is available in online gameplay is a major challenge. In online gameplay only implicit feedback on the appropriateness of the personalisation actions is available, i.e., the AI can only observe the player interacting with the game, while not being provided with labels on the player’s affective state, and explicitly asking the player is too intrustive for the player to fully enjoy the game. In this demo, we show how game personalisation techniques can leverage novel computer visionbased techniques to unobtrusively infer player experiences automatically based on facial expression analysis. Specifically, we employ algorithms for tailoring the affective game experience to the individual user that (1) leverages the proven INSIGHT facial expression recognition SDK as a model of the users affective state1, and (2) employ this model for guiding the online game personalisation process. Specifically, we use the emotion anger versus happiness (or neutral) to measure user experience, and adapt the game accordingly. These algorithms have been previously published in [2]. In this demo, participants can play the video game INFINITE MARIO BROS, while it is being personalised using an AI. The AI typically decreases challenge levels of certain game elements when the face of the user shows anger, and typically increases challenge levels of these game elements when the face of user is neutral or looks happy.